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© Picture coding method. 

© Motion vectors are determined in each frame of 
the input image data, and optimum motion vectors 
characterizing the motion are extracted from the 
determined motion vectors, a three-dimensional 
block is composed in space and in time along the 
extracted optimum motion vectors, and the com- 
posed three-dimensional block is block-coded by 
orthogonal transform or the like, according to the 
disclosed picture coding method. 
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PICTURE CODING METHOD 



BACKGROUND OF THE INVENTION 



1. Held of the Invention 

The present invention relates to a picture cod- 
ing method for coding picture signals at high effi- 
ciency. 



2. Description of the Prior Art 

The transmission speed of digitized picture sig- 
nals has exceeded 100 Mbps, and it is difficult to 
transmit such signals directly by using, the existing 
communication lines from the viewpoints of the 
transmission capacity and transmission cost. The 
picture coding is a technique for removing the 
redundancy of picture signals and reducing the 
transmission speed, and various methods and ap- 
paratuses had been proposed in the past. A repre- 
sentative example of orthogonal-transform coding 
is described below. 

Fig. 6(a) is an example of a conventional block 
coding method, showing a block diagram of a 
three-dimensional orthogonal transform coding ap- 
paratus, in which numeral 61 is a three-dimensional 
blocking circuit, 62 is a orthogonal transform cir- 
cuit, and 63 is a quantizer. 

Figs. 6(b), (c) are explanatory diagrams of con- 
ventional three-dimensional block construction. In 
Fig. 6(b), numeral 1 denotes image data of a first 
frame, 2 is image data of a second frame, 3 is 
image data of a third frame, 11 is a two-dimen- 
sional block in the first frame, 12 is a two-dimen- 
sional block in the second frame, and 13 is a two- 
dimensional block in the third frame. In Fig. 6(c), 
the image data of each frame is drawn as two- 
dimensional plan view in the horizontal and time 
directions, and the frame in the time direction is 
illustrated as an example of up to three frames. In 
Fig. 6(c), numeral 73 denotes part of image data of 
the first frame, 72 is part of image data of the 
second frame, and 73 is part of image data of the 
third frame. 

In thus composed three-dimensional orthogonal 
transform coding apparatus, the operation is de- 
scribed below. 

In Fig. 6, the image input data s(h,v,t) (here h: 
picture element index in the horizontal direction, 
1^h^M, v: picture element index in the vertical 
direction, l^v^N, t: picture element index in the 
time direction, l^t^T) entering the three-dimension- 
al blocking circuit 61 is first divided into mXn two- 
dimensional blocks 71-1 in the first frame, and into 



72-1 in the second frame 72, and so forth, and thus 
at the same two-dimensional position with frame, 
the two-dimensional blocks up to T-th frame are 
summed up thereafter in the time direction to be 

s composed into mXnXT three-dimensional blocks. 

The three-dimensional blocks are orthogonally 
transformed in the orthogonal transform circuit 62. 
The transform coefficient which is the output of the 
orthogonal transform circuit 62 is quantized in the 

w quantizer 63 so that the quantizing bit allotment 
may be more in the coefficient of low sequence 
and be less in the coefficient of high sequence, 
and the guantized transform -coefficient is later de- 
livered as coding output. Generally, a natural pic- 

75 ture is high in the correlation among the picture 
elements in the horizontal, vertical and time direc- 
tions, and orthogonal transform coefficient is large 
in energy in low sequency and small in high 
sequency. Therefore, when quantizing, if small bits 

20 are assigned to the coefficient of high sequency, 
degradation is not obvious visually, and it is possi- 
ble to compress while suppressing the picture 
quality degradation (for example, see chapter 12 of 
"Digital coding of waveform" by Jayant et al., pub- 

25 lished by Academic Press/Prentice-Hall). 

In such conventional structure including three 
dimensions of the horizontal, vertical and time di- 
rection, in the case of a widely moving picture, the 
correction in the time direction is lowered and the 

30 efficiency is poor. 



SUMMARY OF THE INVENTION 

35 It is hence a primary object of the invention to 

present a picture coding method capable of realiz- 
ing a high compression rate even in a widely 
moving picture in the light of the conventional 
problems discussed above. 

40 To achieve the above object, the invention 

presents a picture coding method comprising the 
steps of: 

dividing the input T-frame picture data s(h,v,t) (here 
h: picture element index in the horizontal direction, 

45 1^h^M, v: picture element index in the vertical 
direction, l^v^N, t: picture element index in the 
time direction, 1£t£T) into two-dimensional blocks 
sb(x,y,1) (where x: block coordinate in the horizon- 
* tal direction in frame, 1^x^M/m, y: block coordinate 

so in the vertical direction in frame, l^y^n/n) of m- 
(horizontal direction)Xn (vertical direction) picture 
elements each in the first frame, that is, s(h,v,1), 
obtaining the two-dimensional blocks sb(x,y,2) in 
the size of mXn most closely correlated with said 
sb(x,y,1) on the second frame s(h,v,2), and simulta- 
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neously determining the horizontal motion vector 
mvh(x,y,2) showing the motion quantity in the hori- 
zontal direction of said sb(x,y,2) on the basis of 
said sb(x,y,1) and the vertical motion vector mvv- 
(x,y f 2) showing the motion quantity in the vertical 
direction, 

calculating again the optimum motion vectors 
mvhr(x,y,2), mwr(x,y,2) from the motion vectors 
mvh(x,y,2), mw(x,y,2) of each block in the frame 
with respect to each block, 

adding the motion quantities mvhr(x,y,2), mvvr- 
(x,y,2) of said motion vectors to obtain s(h + mvht- 
(t). v + mvvt(t),t) and cutting out the two-dimensional 
block composed of mXn picture elements on the 
second frame to obtain sb (x,y,2), 
determining the two-dimensional block sb(x,y,3) 
and motion vectors mvh(x,y,3), mvv(x,y.3) in the 
size of mXn most closely correlated with said sb - 
(x,y,2) on the third frame s(h,v,3), 
calculating again, in each block, the optimum mo- 
tion vectors mvhr(x,y,3), mvvr(x,y,3) from the mo- 
tion vectors mvh(x,y,3), mvv(x,y,3) of each block in 
the frame, from the motion quantity between the 
two-dimensional blocks sb'(x,y,2) and sb(x,y,3) ob- 
tained above, adding mvhr(x f y,3), mwr(x,y,3) to the 
motion quantity of said motion vectors to obtain s- 
(h-mvht(t), v-mvvt(t),t), and cutting out a two-dimen- 
sional block composed of mXn picture elements on 
the third frame to obtain sb'(x,y,3), 

Operating thereafter similarly up to the T-th 
frame s(h,v,T) to obtain a two-dimensional block 
sb (x.y.t) in each frame, 

composing a three-dimensional block of said two- 
dimensional blocks sb(x,y,1), sb'(x,y,2) to sb'(x,y,T) 
among said T frames, and 

block-coding at least part of said three-dimensional 
block. 

The invention having such construction, is ca- 
pable of keeping high the correlation in the time 
direction, so that a high compression ratio may be 
realized even in widely moving pictures. 

BRIEF DESCRIPTION OF DRAWINGS 

Fig. 1 is an explanatory diagram of picture cod- 
ing method showing a first embodiment of the 
invention, 

Fig. 2 is an explanatory diagram of picture cod- 
ing apparatus in the first embodiment of the 
invention, 

Fig. 3 is an explanatory diagram of picture cod- 
ing method showing a second embodiment of 
the invention, 

Fig. 4 is an explanatory diagram of picture cod- 
ing method showing a third embodiment of the 
invention, 



Fig. 5 is an explanatory diagram of picture cod- 
ing method showing a fourth embodiment of the 
invention, and 

Fig. 6 is a block diagram of a three-dimensional 

5 orthogonal transform coding apparatus. 

1 ... image data of a first frame, 2 ... image 
data of a second frame, 3 ... image data of a third 
frame, 4 ... motion vector between the first frame 
and second frame, 5 ... is a motion vector between 

70 the second frame and third frame, 6 ... three- 
dimensional block coding circuit for motion com- 
pensation, 7 ... orthogonal transform circuit, 8 ... 
quantizer, 11 ... two-dimensional block in the first 
frame, 12 ... two-dimensional block in the second 

75 frame, 13 two-dimensional block in the third frame, 
14 ... frame memory, 15 ... motion detection circuit, 
16 ... address generation circuit, 17 ... CPU. 21 ... 
part of image data of the first frame, 22 ... part of 
image data of the second frame, 23 ... part of 

20 image data of the third frame, 24 -29,31 - 42 ... 
motion vector, 61 ... three-dimensional block cod- 
ing circuit, 62 ... orthogonal transform circuit, 63 ... 
quantizer, 71 ... part of image data of the first 
frame, 72 ... part of image data of the second 

25 frame. 73 ... part of image data of the third frame. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

30 

Referring now to the drawings, some of the 
preferred embodiments are described in detail be- 
low. 

Fig. 1 is an explanatory diagram of a picture 

35 coding method in a first embodiment of the inven- 
tion, and Fig. 2 is an explanatory diagram of a 
picture coding apparatus in the first embodiment. 
In Fig. 1(a), numeral 1 denotes image data of a first 
frame, 2 is image data of a second frame, 3 is 

40 image data of a third frame, 4 is a motion vector 
between the first frame and second frame, 5 is a 
motion vector between the second frame and third 
frame, 11 is a two-dimensional block in the first 
frame, 12 is a two-dimensional block in the second 

45 frame, and 13 is a two-dimensional block in the 
third frame. Fig. 1(b) is an explanatory diagram 
drawing the picture data of each frame in Fig. 1(a) 
as a two-dimensional plan view in the horizontal 
and time directions, and the frames in the time 

so direction are drawn up to three frames as an exam- 
ple. In this diagram, numeral 21 denotes part of 
image data of the first frame, 22 is part of image 
data of the second frame, and 23 is part of image 
data of the third frame. 

55 Fig. 2(a) shows an example of image coding 

apparatus, in which numeral 6 is a three-dimen- 
sional block coding circuit for motion compensa- 
tion, 7 is an orthogonal transform circuit, and 8 is a 
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quantizer. Fig. 2(b) is a structural diagram of the 
motion compensation threedimensional block cod- 
ing circuit 6, which comprises a frame memory 14, 
a motion vector detection circuit 15, an address 
generation circuit 16, and a CPU 17. 

The input image data is fed into the motion 
compensation three-dimensional block coding cir- 
cuit 6, and is transformed into three-dimensional 
blocks in the following method. The three-dimen- 
sional block image data is orthogonally trans- 
formed, quantized and send out to the transmission 
line in the same manner as in the prior art. 

Fig. 2(b) is a structural diagram of the motion 
compensation three-dimensional block coding cir- 
cuit 6. The image data is written into the frame 
memory 14, and first the motion vector is detected 
by the motion vector detection circuit 15. Next, the 
detected motion vector is fed into the CPU, which 
calculates the beginning address h f v of each two- 
dimensional block according to the motion vector, 
and delivers to the address generation circuit 16. 
The address generation circuit 16 increases the 
beginning address h,v of each two-dimensional 
block, and generates the addresses of h to h + m, v 
to v + n. Therefore, as the CPU continuously gen- 
erates the beginning addresses for the portion of T 
frames, the image data of three-dimensional blocks 
of mXnXT picture elements are sent out from the 
frame memory 14 to the orthogonal transform cir- 
cuit 7. The image coding apparatus shown in Figs. 
2(a), (b) may be commonly used in all embodi- 
ments of the invention by modifying only the pro- 
gram of the CPU. Therefore, in the succeeding 
embodiments, explanations are given while refer- 
ring to the flow chart of the CPU. 

First referring to Fig. 1(b), the detection meth- 
od of motion vectors 24, 25 is described. Of the 
input image data, the reference frame is supposed 
to be the image data 21 of the first frame, which is 
divided into mXn two-dimensional blocks 21-1 to 

21- 5. These two-dimensional blocks are expressed 
as sb(1,1,1) to sb(5,1,1) (where, in sb( x,y,t) x: 
block coordinate in the horizontal direction in 
frame, 1^x£M/m, y: block coordinate in the vertical 
direction in frame, 1^y^N/n, t: frame index, 1£t£T). 

The two-dimensional block 21-1 on the first 
frame is matched in block to the second frame as 
well known in the general image processing tech- 
nology, and the mXn two-dimensional block frame 

22- 1 in the second frame having the closest cor- 
relation, that is, sb(1,1,2) is obtained. The moving 
distance in the horizontal and vertical directions at 
this time is assumed to be the motion vector 24 of 
the first frame, and is expressed as the horizontal 
motion vector mvh(1,1,2) showing the motion ex- 
tent in the horizontal direction, and the vertical 
motion vector mw(1 ,1 ,2) showing the motion extent 
in the vertical direction (where, in mvh(x,y,t), mvv- 



(x.y.t), x: block coordinate in the horizontal direction 
in frame, 1^x^M/m, y: block coordinate in the verti- 
cal direction in frame, 1^y^N/n, t: frame index, 
1£t£T). As for the third frame, on the basis of the 
5 mXn two-dimensional block 22-1 detected in the 
second frame, a similar motion vector 25 is ob- 
tained In the embodiment descriptions hereinafter, 
for the sake of simplicity, the motion extent in the 
vertical direction mvv(x,y,t) =0 is supposed. 
io Incidentally, in the case of block coding such 

as orthogonal transform, as shown in the prior art, 
when three-dimensional blocks are composed as 
71-1, 72-1, 73-1 as shown in Fig. 6(c), if such a 
motion as in Fig. 2 should be detected, it is 21-1, 
75 22-1 and 23-1 that are actually operating, and 
hence the correlation in the time direction is lost 
and the compression efficiency is lowered. There- 
fore, in this embodiment, the three -dimensional 
blocks are composed along the motion vectors 24, 
20 25 detected as shown in Fig. 1(b). That is, if the 
two-dimensional block of the first frame is sb- 
(1,1,1), then the two-dimensional block of the sec- 
ond frame is sb(1,1,2) and the two-dimensional 
block of the third frame is sb(1,1,3). 
25 It is Fig. 2(c) that shows a flowchart of the CPU 

in the three-dimensional block composing method 
mentioned above. First, in the CPU, after feeding 
the motion vectors mvh(x,y,t), mvv(x,y,t), the two- 
dimensional block beginning address h,v, and the 
30 frame index t are initialized. In the first frame, since 
motion vector is not used. h,v are directly deliv- 
ered. In the second and subsequent frames, the 
optimum motion vectors in each block mvhr(x,y,t), 
mwr(x,y f t) are blocked as detected motion vectors 
35 mvh(x,y,t), mw(x,y,t). Therefore, in the second 
frame, adding motion vectors mvh(x,y,2), mvv- 
(x,y,2) to the beginning address h,v, h + mvh(x,y,2), 
v + mvv(x,y,2) are delivered to the address genera- 
tion circuit. In the third frame, furthermore, adding 
40 motion vectors mvh(x,y,3), mvv(x,y,3) to the begin- 
ning address to obtain h + mvh(x,y,2) + mvh(x,y,3), 
v + mvv(x,y f 2) + mvv(x,y,3). Thereafter, operating 
similarly up to the T-th frame, nXmXT three-dimen- 
sional blocks are formed. This operation is effected 
45 in all blocks within the screen. 

This three-dimensional block is displayed on a 
system of three-dimensional coordinates in Fig. 1- 
(a), and the three-dimensional block is composed 
of 11, 12, 13 in Fig. 1(a). The three-dimensional 
so block composed in the technique as shown in Fig. 
1, Fig. 2 is maintained in the correlation in the time 
direction, and therefore by block-coding the three- 
dimensional block, even a rapidly moving picture 
can be compressed and coded efficiently. 
55 Incidentally, when the three-dimensional block 

is composed as shown in Fig. 1, a vacancy or 
overlapping of blocks in a reverse case may be 
formed at a location where the motion vectors 
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differ, as shown between two-dimensional blocks 
23-3 and 23-4 in Fig. 1(b). In the event of a 
vacancy, the image data is interpolated by any 
known interpolation technique, such as replace- 
ment with signal value of upper or lower line, 
holding of previous value, holding of succeeding 
value, mean interpolation, and interpolation using 
motion vector. Or overlap of blocks may be cleared 
by replacing with either block decoding value. 

Fig. 3 is an explanatory diagram of picture 
coding method showing a second embodiment of 
the invention. Fig. 3(a), same as Fig. 1(b). is plot- 
ting of the data of each frame as two-dimensional 
plan view in the horizontal and time directions, in 
which numerals 21, 22, 23 are same as in the first 
embodiment. Fig. 3(b) is a CPU flowchart for realiz- 
ing this embodiment. 

In this embodiment, when the direction of the 
motion vector within same frame is equal by the 
number over a certain threshold value vth and the 
magnitude of motion vector is included within a 
certain range vwh, optimum vectors mvhr(t), mvvr- 
(t)(where t is the frame index) are obtained by one 
each in the horizontal and vertical directions within 
the frame, where vth and vwh are experimentally 
determined. 

Suppose the motion vector is obtained same 
as in Fig. 1(b), then, in Fig. 3(a), the motion vectors 
26, 27 in Fig. 1(b) are replaced by the optimum 
motion 24, and 28, 29 by 25. 

In the flowchart shown in Fig. 3(b), what is 
different from Fig. 2(c) is that the optimum motion 
vector is not directly replaced by the detected 
motion vector, but is calculated from the motion 
vector in the following techanique. First, a histo- 
gram of all blocks is plotted as for the magnitude 
of the motion vector mvh(x,y,t), mvv(x,y,t) in the 
frame. Next, in this histogram, the one having the 
highest frequency is determined as the optimum 
motion vector mvhr(t), mvvr(t). Or if the frequency 
distribution is uniform and it cannot be determined 
by the frequency alone, the optimum motion vector 
is determined as the mean excluding i pieces from 
the maximum value and j pieces from the minimum 
value, of the motion vectors of each block in the 
frame. This exclusion of i pieces and j pieces from 
the maximum and minimum respectively is intend- 
ed to eliminate detecting errors in the motion de- 
tection, and the number of i and j should be 
experimentally determined by each detecting meth- 
od. If there is no detecting error, i and j may be 
zero. 

Consequently, on the basis of the optimum 
motion vector mvhr(t), mvvr(t), in the same opera- 
tion as in the first embodiment, the beginning ad- 
dress of the two-dimensional block is generated, 
and two-dimensional blocks from the first to the T- 
th frame are sequentially read out from the frame 



memory, thereby composing a three-dimensional 
block including mXnXT picture elements. 

Such manner of composing the three-dimen- 
sional block is optimum when the entire screen is 

5 panning. Besides, within the frame, all motion vec- 
tors are equal, there is no problem such as va- 
cancy as experienced in Fig. 2. 

Fig. 4 is an explanatory diagram of picture 
coding method showing a third embodiment of the 

70 invention. Same as Fig. 3(a). this is also a two- 
dimensional plan plotting of the frame data in the 
horizontal and time directions, in which numerals 
21, 22 and 23 are meant same as in the first 
embodiment. Fig. 4(a) is a two-dimensional block 

75 diagram of the first frame, showing the result of 
motion detection by using this block as the refer- 
ence, in which 22-1 \ 23-1 ' are the second and 
third frame two-dimensional blocks determined on 
the basis of the first frame two-dimensional block 

20 21-1, 24,31, and 33 are motion vectors between the 
first and second frames, and 25, 32 and 34 are 
motion vectors between the second and third 
frames. 

Fig. 4(b) is an explanatory diagram showing the 
25 method of composing a three-dimensional block by 
the motion vectors calculated again by employing 
the method shown in the embodiment below. Fig. 
4(c) is an explanatory diagram showing the result 
of detection of motion vector different from Fig. 4- 
30 (a). Figs. 4(d), (e) are flowcharts of the CPU for 
realizing this embodiment. Figs. 4(f), (g) are exam- 
ples of the detected motion vector and the motion 
vector calculated again by the technique of this 
embodiment. 

35 In the third embodiment, what is different from 

the second embodiment is the composing method 
of three-dimensional block. The motion vectors de- 
tected in the same manner as in the second em- 
bodiment are not mostly facing the same direction 

40 in the frame, but as shown in Fig. 4(a). as advan- 
cing to the second and third frames, the motion 
vectors are gradually centering around the two- 
dimensional blocks 22-3', 23-3'. In such case, 
since the directions of the motion vectors are dif- 

45 ferent. optimum motion vectors cannot be uniform- 
ly determined in the frame as in the second em- 
bodiment. Or if determined uniformly and the 
three-dimensional block is composed by using 
such vectors, the correlation in the time direction 

so cannot be enhanced. Incidentally, when the three- 
dimensional block is composed along the motion 
vectors shown in Fig. 4(a), the correlation in the 
time direction is heightened, but a vacancy or a 
portion incapable of coding is formed as shown in 

55 the diagram between the blocks 23-1 and 23-2 . 
Such motion vectors are determined at the time of 
zooming or the like. 

In this embodiment, accordingly, by treating 



5 



9 



EP 0 405 842 A2 



10 



the motion vectors as shown in Fig. 4(e), the opti- 
mum motion vectors are determined. In the first 
place, the central blocks 21-3, 22-3 t 23-3 as the 
center of motion are determined as follows. The 
number hz(x,t) which is the motion vector mvh- 
(x,y,t) = 0 showing the motion extent in the hori- 
zontal direction is calculated in the relation of 
0^y<M/m, and the block coordinate in the horizon- 
tal direction having the greatest number hz(x.t) is 
regarded as the block coordinate cx in the horizon- 
tal direction of the central block, and 
the number vz(y,t) which is the motion vector mvv- 
(x,y,t) = 0 showing the motion extent in the vertical 
direction is calculated in the relation of O^x^N/n, 
and the block coordinate in the vertical direction 
having the greatest number vz(y,t) is regarded as 
the block coordinate cy in the vertical direction of 
the central block. 

Fig. 4(f) is an example of detected horizontal 
motion vector. When the above method is applied 
to this example, we obtain cx = 3 because (hz(1 ,t)- 
hz(2,t)hz(3,t),hz(4,t),hz(5,t) = (0,0,5,1,0). 

Or, in other method, of the motion vectors 
mvh(x,y,t) showing the motion extent in the hori- 
zontal direction, the number hcz(x.t) having zeros 
continuously is calculated in the frame t as for the 
block in the vertical direction, and the block coordi- 
nate cx in the horizontal direction having the great- 
est number hcz(x s t) is regarded as the block co- 
ordinate in the horizontal direction of the central 
block, and 

of the motion vectors mvv(x,y,t) showing the motion 
extent in the vertical direction, the number vcz(y.t) 
having zeros continuously is calculated in the 
frame t as for the block in the horizontal direction, 
and the block coordinate cy in the vertical direction 
having the greatest number vcz(y.t) is regarded as 
the block coordinate in the vertical direction of the 
central block. 

In the latter method, the structure of the ap- 
paratus is somewhat complicated as compared 
with the former method, but if there is error detec- 
tion of motion vector, it is effective since the vec- 
tors of the motion extent = 0 determined by error 
detection can be eliminated. 

Representative motion vectors 35, 36, 39, 40 in 
Fig. 4(b) are determined in the following method. 
Of the motion vectors in each frame, a histogram is 
plotted in a range of 1 <y<N/n of the motion vectors 
mvh(cx-x\y,t), mvh(cx + x",y,t) in a block apart from 
the central block by block coordinates x\ x each, 
and the ones having the highest frequency in the 
histogram are supposed to be representative mo- 
tion vectors mvhr1(t), mvhr2(t) in the horizontal 
direction, respectively, and of the motion vectors in 
each frame, a histogram is plotted in a range of 
Kx<M/m of the motion vectors mvvfc.cy-y'.t), 
mvh(x,cy + y ",t) in a block apart from the central 



block by block coordinates y'.y" each, and the 
ones having the highest frequency in the histogram 
are supposed to be representative motion vectors 
mvvr1(t), mvvr2(t) in the vertical direction, respec- 
5 tively. 

Or, in other method, if there is no deviation in 
the frequency deviation and it cannot be deter- 
mined by the frequency alone, the mean value 
excluding i pieces from the maximum value of the 

70 motion extent and j pieces from the minimum value 
is obtained as the representative motion vector. 
This exclusion of i pieces and j pieces from the 
maximum and minimum respectively is intended to 
eliminate the detection errors, and the number of i 

75 and j should be determined experimentally in each 
detecting method. If there is no detecting error, i 
and j may be zero. 

Referring now to Fig. 4(f), a method of using 
histogram is explained below. In terms of x' , x , the 

20 end points of the screen are expressed as 
x ' = x " = 2. Since cx-3, individual histograms are 
calculated in a range of 1<y<5 of mvh(3-2,y,t), 
mvh(3 + 2,y,t). In the histogram, in mvh(1,y,t), -3 is 
found twice, -1 once, and in mvh(5,y,t), 3 twice and 

25 2 once, and hence mvhr1(t) = -3. mvhr2(t) = 3 are 
obtained. In Fig. 4(b), 35 is mvhr1(2), 39 is mvhr2- 
(2); 36 is mvhr1(3), and 40 is mvhr2(3). 

Employing thus obtained central block and re- 
presentative motion vectors, the optimum motion 

30 vectors mvh'fx.y.t), mvv'(x,y,t) in the horizontal and 
vertical directions of each block are calculated as 
follows. 

mvh'(x,y ,t) = mvhrt (t)*(cx-x)/x' w (where x<cx) (1 ) 
mvh'(x,y,t) = mvhr2(t)"(x-cx)/x'' (where x> = cx) 
35 (2) 

mvv'(x,y,t = mvvrl (t)*(cy-y)/y /where y<cy) (3) 
mw'(x,y,t) = mvvr2(t)*(y-yc)/y" (where y> = cy) 
(4) 

where cx, cy are block coordinates of the central 
40 block, mvhr1(t), mvhr2(t) t mvvrl (t), mvvr2(t) are re- 
presentative motion vectors, x' is the block coordi- 
nate when obtaining the representative motion vec- 
tor mvhr1(t), and hereinafter x" corresponds to 
mvhr2(t), y' to mwr1(t), and y" to mvvr2(t). 
45 It is Fig. 4(g) that shows an example of calcula- 

tion of mvht'(x,y,t) by applying thus obtained cx = 3, 
x' = x" = 2, mvhr1(t) = -3, mvhr1(t) = 3 to the example 
in Fig. 4(f). 

Otherwise the following equations may be also 

so employed. 

mvh^x.y^-mvhriancx-x^/x^Jwhere x<cx) (5) 
mvh'(x,y,t) = mvhr2(tnx-cx) 2 /x ,/2 (where x>=cx) 
(6) mvv'(x.y.t) = mvvrl (t)*(cy-y) 2 /y' 2 (where y<cy) 
(7) mvv'(x,y,t) « mvvr2(t)*(y-cy) 2 /y" 2 (where 

55 y>=cy) (8) whether equations (1) to (4) or 

equations (5) to (8) should be used may be deter- 
mined in consideration of the compression effi- 
ciency in application to pictures. 
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Next, according to the optimum motion vectors 
mvh'(x,y.t), mvv'(x,y,t), in the same operation as in 
the first and second embodiments, the beginning 
address of the two-dimensional block is generated, 
and the two-dimensional blocks from the first to the 
T-th frame are sequentially read out from the frame 
memory, and a three-dimensional block comprising 
mXnXT picture elements is composed. In the fol- 
lowing technique, as shown in Fig. 4(b), it is possi- 
ble to compose a three-dimensional block main- 
taining the correlation in the time direction, having 
no vacancy among blocks. Therefore, by block- 
coding of the three-dimensional block, a high com- 
pression efficiency is realized. 

In the third embodiment, meanwhile, when the 
picture zooms up, for example, the motion vector 
as shown in Fig. 4(c) is detected. When such 
vector is detected, the motion vector is detected 
again in the time direction on the basis of the latest 
frame. In other words, supposing the reference 
frame to be the T-th frame, the motion is detected 

in the sequency of frame T-1 3, 2, 1. As a 

result, the motion vector as shown in Fig. 4(c) 
becomes a same motion vector as shown in Fig. 4- 
(a), equivalently. Therefore, in the same manner as 
in the third embodiment, an effective compression 
is realized. 

Fig. 5 is an explanatory diagram of a picture 
coding method showing a fourth embodiment of 
the invention. Fig. 5, same as Fig. 3 and Fig. 4, is a 
two-dimensional plan plotting of the image data of 
each frame in the horizontal and time directions, in 
which numerals 21 . 22, 23 are same as in the third 
embodiment. Same as in Fig. 4, in Fig. 5, too, the 
horizontal direction of the picture data of one frame 
is composed of five blocks, and 21-1, 21-5 denote 
the blocks at the end of the picture data of the first 
frame, 22-1, 22-5 those of the second frame, and 
23-5 of the third frame. 

Supposing to perform three-dimensional bloc- 
king, as shown in the second embodiment, in each 
frame by using the optimum motion vectors of 41 , 
42, Fig. 5(a) is a diagram showing the blocking 
result, in which 21-2 to 21-5 are normally blocked, 
but 21-1, 22-1 lack image data in the third frame 
and cannot be blocked. Meanwhile, 22-5, 23-5, 
having no reference two-dimensional block in the 
first frame, cannot be blocked, and these pieces of 
image data cannot be coded in this state, and will 
be lost. 

Incidentally, in this embodiment, as shown in 
Fig. 5(b) as 21-0 and 21-6, three-dimensional 
blocks conforming to the conventional method with- 
out using optimum motion vectors are composed, 
and are coded separately from the three-dimen- 
sional block shown in the second embodiment, and 
transmitted. At this time, 21-0, 21-6 are equivalent 
to 21-1, 21-5, respectively, in the first frame. By 



this method, the end points of the screen may be 
also coded. 

Fig. 5(c) is an explanatory diagram for com- 
pensating the second, that is, insufficient picture 

5 elements for coding the end points, by holding the 
picture elements in the same frame. In the dia- 
gram,* as indicated by arrow 50, 22-5 and 23-5 are 
shifted to the other end of the frame, thata is, 22-5 
is adjacent to 22-1 and 23-5 to 23-1. Next, by 

70 blocking 21-1 by using motion vectors 41, 42, a 
three-dimensional block is composed. And then 
block coding is carried out. When decoding, in the 
reverse procedure, 22-5 and 23-5 are returned to 
the original positions to obtain decoded images. In 

75 this method, since it is not necessary to send the 
end points of screen separately, it is possible to 
code by using a smaller quantity of data as com- 
pared with the first method. 

Figs. 5(d). (e), (f) are explanatory diagrams of 

20 the method of compensating the third, that is, in- 
sufficient picture elements for coding the end 
points of the screen by holding of picture elements 
of other frames. Supposing, here, to compose a 
three-dimensional block by using the method 

25 shown in the second embodiment as in Fig. 5(d). in 
which the dotted line shows the block boundary, 
each block is composed along the motion vectors 
indicated by thick lines in the diagram. In Fig. 5(d), 
same as in Fig. 5(b), regions incapable of coding 

30 are present in screen end points 22-0. 23-0, 22-6. 

22- 7. This is because the three-dimensional block 
is mutually overlapped after the second frame. 

Accordingly, in Fig. 5(e), of the motion vectors 
detected as in Fig. 5(d), new blocks 21-0. 21-6 are 

35 composed by using the motion vectors, 24,25,26, 
and 27 determined on the basis of the blocks 21-1, 
21-5 including the end points of the image of the 
first frame. The reference frame at this time is the 
frame reverse to Fig. 5(c) timewise, that is, 23. Fig. 

40 5(e) is an example of composing one three- dimen- 
sional block with three frames. In this case, the 
reference two-dimensional blocks are third frame 

23- 0, 23-6. This is because the number of uncoded 
picture elements is most in the third frame 23. 

45 Next, using motion vectors 24 to 27, a three-dimen- 
sional block is composed. Fig. 5(f) is an explana- 
tory diagram showing the detail of the composition 
of the three-dimensional block shown in Fig. 5(e). 
In the diagram, 21 to 23. 21-0, 23-0 are same as in 

50 Fig. 5(e), and the blocks corresponding to 21-0 to 
23-0 in Fig. 5(e) are magnified. Now, on the basis 
of the two-dimensional block 23-0 including the end 
points of the third frame, using the motion vectors 
25, 24 in Fig. 5(e), a three-dimensional block is 

55 composed, but in the second frame and first frame, 
there is no image data of the portion corresponding 
to 51, 52, respectively. Accordingly, 52 is replen- 
ished with part of 23-0 indicated by 53, and 51 with 
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part of 52 and 22-0, as indicated by arrows, there- 
by composing a three-dimensional block, and this 
three- dimensional block is block-coded as a new 
block. In this way, the correlation in the time direc- 
tion is maintained within the block, and therefore as 
compared with the method shown in Fig. 5(a), (b), 
it is possible to code the end point block of the 
picture at high compression rate and high picture 
quality. 

In the fourth embodiment, meanwhile, the first 
and second methods were explained by combining 
with the second embodiment and the third method 
in combination with the third embodiment, but 
these are not limitative. 

Incidentally, in the foregoing embodiments, all 
motions in the vertical direction were supposed to 
be zero, but this is not limitative, and exactly the 
same method in the horizontal method may be 
applied also to the vertical direction. 

As examples of block coding, in the embodi- 
ments, orthogonal transform was explained, but this 
is not limitative, and the invention may be equally 
applied to any coding method by dividing into 
blocks, such as vector quantizing, and DPCM in 
block units. 



Claims 

1 . A picture coding method comprising the steps 
of: 

dividing the input T-frame picture data s(h,v,t) (here 
h: picture element index in the horizontal direction, 
1^h<M, v: picture element index in the vertical 
direction, 1^ V ^N, t: picture element index in the 
time direction, 1<t£T) into two-dimensional blocks 
sb(x,y,1) (where x: block coordinate in the horizon- 
tal direction in frame, 1£x^M/m, y: block coordinate 
in the vertical direction in frame, 1^y^N/n) of m- 
(horizontal direction) X n(vertical direction) picture 
elements each in the first frame, that is, s(h,v,1), 
obtaining the two-dimensional blocks sb(x,y,2) in 
the size of mXn most closely correlated with said 
sb(x,y,1) on the second frame s(h,v t 2), and simulta- 
neously determining the horizontal motion vector 
mvh(x f y,2) showing the motion quantity in the hori- 
zontal direction of said sb(x,y,2) on the basis of 
said sb(x,y,1) and the vertical motion vector mvv- 
(x,y,2) showing the motion quantity in the vertical 
direction, 

calculating again the optimum motion vectors 
mvhr(x,y,2), mvvr(x,y t 2) from the motion vectors 
mvh(x,y,2), mvv(x,y,2) of each block in the frame 
with respect to each block, 

adding the motion quantities mvhr(x,y,2), mvvr- 
(x,y,2) of said motion vectors to obtain s(h + mvht- 
(t), v + mwt(t),t) and cutting out the two-dimensional 
block composed of mXn picture elements on the 



second frame to obtain sb'(x,y,2), 
determining the two-dimensional block sb(x,y,3) 
and motion vectors mvh(x,y,3), mvv(x,y,3) in the 
size of mXn most closely correlated with said sb'- 

5 (x,y,2) on the third frame s(h,v,3), 

calculating again, in each block, the optimum mo- 
tion vectors mvhr(x,y,3), mwr(x,y,3) from the mo- 
tion vectors mvh(x,y,3), mvv(x,y,3) of each block in 
the frame, from the motion quantity between the 

70 two-dimensional blocks sb'(x,y,2) and sb(x,y,3) ob- 
tained above, adding mvhr(x,y,3), mvvr(x,y,3) to the 
motion quantity of said motion vectors to obtain s- 
(h-mvht(t), v-mvvt(t),t), and cutting out a two-dimen- 
sional block composed of mXn picture elements on 

75 the third frame to obtain sb'(x,y,3), 

Operating thereafter similarly up to the T-th frame 
s(h,v,T) to obtain a two-dimensional block sb'(x,y,t) 
in each frame, 

composing a three-dimensional block of said two- 
20 dimensional blocks sb(x,y,1), sb'(x,y,2) to sb'(x,y,T) 
among said T frames, and 

block-coding at least part of said three-dimensional 
block. 

2. A picture coding method of claim 1, wherein said 
25 detected motion vectors mvh(x.y't), mvv(x,y,t) are 

used as said optimum motion vectors mvhr(x,y,t), 
mvvr(x,y,t). 

3. A picture coding method of claim 1, wherein said 
optimum motion vectors mvhr(x,y,t), mvvr(x,y,t) are 

30 provided by one type in each frame. 

4. A picture coding method of claim 3, wherein said 
optimum motion vectors mvhr(t), mvvr(t) are plotted 
in histograms of motion vectors mvh(x,y,t), mvv- 
(x.y.t) of each block in the frame, and the one of 

35 the highest frequency is selected in the histogram. 

5. A picture coding method of claim 3, wherein said 
optimum motion vectors mvhr(t), mvvr(t) are ob- 
tained as the mean by excluding i pieces from the 
maximum value, and j pieces from the minimum 

40 value, of the motion vectors mvh(x,y,t), mvv(x,y,t) 
of each block in the frame. 

6. A picture coding method of claim 1 , wherein the 
central block as the center of motion, representa- 
tive horizontal motion vectors with different signs 

45 mvhr1(t). mvhr2(t), and representative vertical mo- 
tion vectors with different signs mvvrl(t), mwr2(t) 
are obtained from the motion vectors mvh(x,y,t), 
mvv(x,y,t) of each block, and 
optimum motion vectors mvhr(x,y,t), mvvr(x,y,t) are 

so calculated and obtained from the central block and 
the representative motion vectors, mvhr1(t), mvhr2- 
(t), mvvr1(t), mvvr2(t). 

7. A picture coding method of claim 6, wherein the 
number hz(x,t) of which motion vector mvh(x,y,t) 

55 showing the motion extent in the horizontal direc- 
tion is 0 is calculated in frame t as for the block in 
the vertical direction, and the block coordinate in 
the horizontal direction having the greatest number 
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hz(x,t) is regarded as the block coordinate cx in the 
horizontal direction of the central block, and 
the number vz(y.t) of which motion vector mw- 
(x,y f t) showing the motion extent in the vertical 
direction is 0 is calculated in the frame t as for the 
block in the horizontal direction, and the block 
coordinate in the vertical direction having the great- 
est number vz(y.t) is regarded as the block coordi- 
nate cy in the vertical direction of the central block. 

8. A picture coding method of claim 6, wherein the 
number hcz(x.t) with consecutive zeros out of the 
motion vector mvh(x,y,t) showing the motion extent 
in the horizontal direction is calculated within the 
frame t as for the block in the vertical direction, 
and the block coordinate in the horizontal direction 
having the greatest number hcz(x.t) is regarded as 
the block coordinate cx in the horizontal direction 
of the central block, and 

the number vcz(y,t) with consecutive zeros out of 
the motion vector mvv(x,y,t) showing the motion 
extent in the vertical direction is calculated within 
the frame t as for the block in the vertical direction, 
and the block coordinate in the vertical direction 
having the greatest number vcz(y,t) is regarded as 
the block coordinate cy in the vertical direction of 
the central block. 

9. A picture coding method of claim 6, wherein the 
representative horizontal motion vectors mvhrlt), 
mvhr2(t) are plotted in a histogram in a range of 
1<y<N/n of motion vectors mvh(cx-x',y,t), mvh- 
(cx + x',y,t) of the block apart from the central block 
by block coordinates x\ x", and are the ones of the 
highest frequency in the histogram, and 

the representative vertical motion vectors mwr1(t), 
mvvr2(t) are plotted in a histogram in arrange of 
1<x<M/m of motion vectors mvv(x,cy-y ,t), mvh- 
(x.cy+y",t) of the block apart from the central 
block by block coordinates y , y , and are the ones 
of the highest frequency in the histogram. 

10. A picture coding method of claim 6, wherein 
the representative horizontal motion vectors mvhrl- 
(t), mvhr2(t) are the mean values excluding i pieces 
from the maximum value of the motion extent and j 
pieces from the minimum value, of the motion 
vectors mvh(cx-x',y,t), mvh(cx + x",y,t) of the block 
apart from the central block by block coordinates 
x , x , and 

the representative vertical motion vectors mvvr1(t), 
mwr2(t) are the mean values excluding i pieces 
from the maximum value of the motion extent and j 
pieces from the maximum value, of the motion 
vectors mvv(x,cy-y't), mvh(x,cy + y ,t) of the block 
apart from the central block by block coordinates 

y . y • 

11. A picture coding method of claim 6, wherein 
the optimum motion vectors mvh (x.y.t), mvv (x.y.t) 
are generated by calculating as follows: 
mvh'(x,y,t1 = mvhr1 (t)*(cx-x)/x' \vhere x<cx), 



mvh'(x,y,t) = mvhr2(t)*(x-cx)/x" where x> =cx), 
mw (x.y.t) = mvvrl (t)*(cy-y)/y "where y<cy) t 
mw'(x,y.t) = mvvr2(tny-cy)/y" "where y> =cy), 
in terms of the block coordinates cx, cy of the 
5 central block, representative motion vectors mvhrl- 
(t), mvhr2(t), mvvrl (t), mvvr2(t), block coordinate x' 
when obtaining the representative motion vector 
mvhr1(t), x" for mvhr2(t), y' for mvvrl (t), and y° for 
mvvr2(t). 

70 12. A picture coding method of claim 6, wherein 
the optimum motion vectors mvh'(x t y,t), mvv'(x,y,t) 
are generated by calculating as follows: 
mvh'(x,y,t) = mvhr1(t)"(cx-x) 2 /x' 2 (where x<cx), 
mvh'(x,y l t) = mvhr2(t)*(x-cx) 2 /x'' 2 (where x> =cx), 

75 mvv'(x,y ,t) = mvvrl (t)*(cy-y) 2 /y' 2 (where y>cy), 
mvv / (x,y,t) = mvvr2(t)*(y-cy) 2 /y" 2 (where y> =cy), 
in terms of said cx, cy. mvr1(t), mvhr2(t), mwr1(t), 
mvvr2(t), x , x ', y' and y 

13. A picture coding method of claims 1, 2. 3, 6, 
20 wherein said motion vector is determined by as- 
suming the frame to be used as the reference of 
motion vector to be the T-th frame image data s- 
(h.v.T), detecting against the adjacent T-1 frame 
image data s(h,v t T-1), determining the two-dimen- 

25 sional block of the size of mXn having the closest 
correlation, and thereafter similarly detecting the 
motion up to the first frame s(h,v,1). 

14. A picture coding method of claims 1, 2, 3, 6, 
wherein the block containing the end points in the 

30 horizontal or vertical direction, out of the three- 
dimensional blocks, is formed as three-dimensional 
block having the motion vector as zero among 
consecutive T frames, and bLock-coding the three- 
dimensional block. 

35 15. A picture coding method of claims 1, 2, 3, 6, 
wherein if there is no image data in the three- 
dimensional block and it is impossible to form 
block, insufficient picture elements are compen- 
sated by holding picture elements within the same 

40 frame to compose a three-dimensional block, and 
this three-dimensional block is block-coded. 
16. A picture coding method of claims 1, 2. 3, 6, 
wherein if there is no image data in the three- 
dimensional block and it is impossible to form 

45 block, insufficient picture elements are compen- 
sated by holding picture elements in a different 
frame to compose a three-dimensional block, and 
this three-dimensional block is block-coded. 

50 
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